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Fig.  1  General  Structure  of  an  inference  engine. 

Fig.  2  Block  diagram  of  the  infernet  Triangles  represent  input/output  neural  layers. 

trapezoids  represent  input/output  processing  layers,  and  rectangles  represent 
inference  processing  layers.  The  number  of  arrows  between  two  layers  gives  a 
relative  indication  of  the  number  of  connections. 

Fig.  3  An  input  processor  in  the  Infernet 

Fig.  4  An  output  processor  in  the  Infernet 

Fig.  5  An  expanded  block  diagram  of  the  hierarchical  inference  processor  (HIP)  of  the 

Infernet  In  this  figure,  each  connection  arrow  represents  a  vector  of  neural 

signals  and  each  rectangle  corresponds  to  a  neural  layer.  Thus,  Level  1  is 

composed  of  several  layers  while  Levels  2  and  3  consist  of  single  layers.  The 
triangles  for  the  internal  input  and  output  signals  indicate  that  they  can 
undergo  a  transformation  to/from  the  neural  world  of  the  HIP,  depending  upon 
their  desired  form.  Note  that  the  structure  has  no  explicit  memory  blocks  or 

components;  knowledge  is  stored  distributively  throughout  the  HIP  levels. 

Fig.  6  Neural  wave  interaction  within  the  HIP. 

Fig  7  The  relationship  between  the  neuron  inputs  {Kt),  the  neuron  state  p(t),  and 

the  neuron  output  q(t)  in  the  pulse  neuron  modeL  The  structure  of  the  cell  is 

shown  in  (a),  the  set  of  inputs  overlaid  on  a  common  time  axis  in  (b),  the 
state  in  (c),  and  the  resulting  output  sequence  in  (d).  The  state  is  shown  for 

the  casewhen  p(t)  is  reset  to  zero  when  an  output  impulse  is  generated.  Note 
that  the  jumps  in  the  state  correspond  to  the  input  impulses  and  are  scaled  by 
the  interconnection  coefficients  uj(t)> 

Fig  8  Sample  plot  of  r)(t)  illustrating  the  I(r))  and  A(7))  functions  used  in  the 

differential  Hebbian  learning  relationship. 

Fig.  9  General  form  for  a  layer  of  pulse  neurons  in  the  Infernet 

Fig.  10  The  spatial  coordinate  system  used  in  the  optical  data  path.  The  region  within 

the  dashed  box  contains  the  light  distribution  representing  a  neural  variable. 

Fig.  1 1  A  general  space- variant  optical  imaging  system  which  implements  Eq.  (26X 

Fig.  12  The  space- variant  optical  imaging  system  which  implements  the  layer  state 

equation  in  Eq.  (25). 

Fig.  13  The  optical  implementation  of  an  Infernet  layer.  As  in  Fig  12,  the  distances 

between  the  imaging  lenses  Lj,  L2,  and  L^  and  their  respective  input  and  output 

planes  are  2f.  In  the  conventional  optical  processor,  the  distances  between 
the  transform  lenses  L^  and  Lj  and  their  associated  input,  transform,  and 
output  planes  are  f. 

Fig.  14  Simplied  version  of  the  optical  layer  shown  in  Fig  13.  Interconnection  CGH’s 

arc  omitted  for  clarity.  Each  optical  path  represents  the  firing  activity  of  a 

matrix  of  cells. 

Fig.  15  Relationship  of  the  simplified  optical  layer  (Fig  14)  to  the  block  diagram  of 

the  Infernet  (Fig  5). 
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INTRODUCTION 


Symbolic  logic  problems  involve,  in  an  abstract  sense,  a  set  of  data  objects 

(also  known  as  symbols)  and  a  set  of  relationships  describing  the  data 
objects.  The  data  objects  and  relationships  constitute  a  knowledge  base 

which  is  generally  arranged  as  sets  of  facts  and  rules.  A  fact  is  a 

statement  connecting  a  relationship  with  one  or  more  data  objects  such  that 
the  statement  is  always  interpreted  as  being  true.  On  the  other  hand,  a 

rule  is  a  statement  which  defines  a  relationship  using  other  relationships, 

data  objects,  and/or  facts. 

A  symbolic  logic  problem  is  usually  stated  in  the  form  of  one  or  more 
querVs  which  are  questions  concerning  the  relationships  and  the  data 

oi>jects.  The  queries  are  answered  by  applying  logical  inference  to  the 

knowledge  base  of  rules  and  facts.  This  inference  process  generates  a  set 

of  assertions  (inferred  facts)  from  the  knowledge  base.  The  solution  to  the 
queries  therefore  becomes  a  set  of  conclusions  in  the  form  of  data  objects 
which  is  inferred  from  the  set  of  assertions  so  as  to  satisfy  the  queries. 
Symbolic  logic  problems  are  relatively  common.  They  arise  in  areas  such  as 

pattern  recognition,  expert  systems,  and  other  artificial  intelligence 

systems. 

The  general  structure  of  an  inference  machine  is  shown  in  Fig.  1.  It 

accepts  as  inputs  a  set  of  facts  and  a  set  of  rules  from  the  knowledge  base, 

and  one  or  more  queries.  The  output  of  the  inference  machine  is  a  set  of 
specific  conclusions  which  are  logically  inferred  from  the  facts  and  rules 
in  response  to  the  queries. 


Fig.  1  General  structure  of  an  inference  machine. 
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During  the  first  year  of  the  program,  we  designed  and  started  to  investigate 
our  mapped-template  architecture  for  solving  symbolic  inference  problems 
[1,2].  This  approach  focused  on  the  use  of  optical  information  processing 
techniques  to  quickly  search  the  knowledge  base.  However,  we  found  that 

some  of  the  limitations  of  the  mapped-template  system  which  could  be 
overcome  by  modifying  the  architecture.  First,  the  mapped-template 
architecture  is  inherently  not  fault  tolerant.  That  is,  if  any  optical 
obstructions  (such  as  large  dust  particles)  became  lodged  on  the  lenses, 

mirrors,  or  spatial  light  modulators,  they  could  block  the  light  beam 
corresponding  to  a  particular  data  object  and  thus  cause  large  errors. 
Second,  the  number  of  usable  data  objects  was  limited  by  the  size  of  the 

spatial  light  modulators.  Since  current  research  devices  have  only  modest 

space-bandwidth  products,  the  number  of  data  objects  that  could  be 
represented  was  quite  limited.  Third,  the  mapped-template  concept 
implements  deductive  logic,  making  it  necessary  to  program  the  knowledge 
base  into  the  system.  This  makes  the  system  inflexible  to  assimilating  new 
information  (a  problem  shared  by  conventional  expert  systems).  Finally, 
since  the  knowledge  base  is  stored  electronically,  the  architecture  is  very 
dependent  upon  a  digital  computer  for  manipulating  the  knowledge  base.  This 

necessarily  involves  some  searching,  and  so  although  the  mapped-template 
approach  reduces  the  amount  of  electronic  searching  it  does  not  completely 

eliminate  it. 

Because  of  these  limitations,  we  decided  to  consider  the  use  of  neural 

network  models  to  realize  an  optical  inference  machine  architecture.  Neural 
networks  offer  possible  solutions  to  the  above  limitations.  They  offer  a 
significant  amount  of  fault  tolerance.  While  the  size  of  any  neural  network 

in  an  optical  implementation  is  constrained  by  the  spatial  light  modulators, 
the  size  of  the  knowledge  base  is  ultimately  limited  by  the  number  of 

interconnects.  If  holographic  techniques  are  used  to  implement  the 

interconnects,  then  a  large  number  can  be  achieved  in  principle.  With  the 

knowledge  base  stored  optically,  the  electronic  computer  is  only  responsible 
for  controlling  the  optical  system  and  interacting  with  the  user  (and  so  the 

systems  we  developed  during  the  first  year  of  the  program  are  still 
necessary).  Thus,  the  computer  does  not  have  to  directly  manipulate  the 

knowledge  base.  Furthermore,  neural  network  models  offer  the  flexibility  of 
learning  new  information  via  inductive  reasoning,  a  feature  not  shared  by 

the  deductive  mapped-template  architecture. 

We  retain  the  belief  that  optical  inference  machines  should  be  a  hybrid 
composition  of  optical  and  electronic  systems.  Although  electronics  is  a 
well  developed  technology,  it  is  expensive  to  perform  parallel  processing,  a 
natural  ability  of  optics.  During  the  first  year  of  the  program,  we 
developed  a  computerized  control  system  (CCS)  for  controlling  the  optical 
devices  and  acquiring  electronic  measurements.  However,  this  system  lacked 
an  efficient  method  for  electronically  measuring  a  two-dimensional  light 
distribution.  Consequently,  during  this  second  year,  we  developed  an 
automated  two-dimensional  photocalibration  system  for  our  CCS.  It  offers 
flexibility  of  alignment  and  fast,  convenient  operation. 
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APPROACH 


In  general,  an  inference  machine  can  be  thought  of  as  a  fancy  signal 
processing  box  with  inputs  and  outputs.  Everything  outside  of  the  box  is 

considered  the  *real  world*  or  its  environment.  The  queries  to  the  system 

are  signals  that  originate  in  the  real  world  and  enter  the  inference  machine 
through  its  inputs.  In  the  same  way,  the  conclusions  are  signals  that  enter 
the  real  world  via  the  system’s  output.  This  structure  allows  feedback  in 

the  real  world  so  past  conclusions  may  affect  future  queries. 

When  the  machine  is  constructed  using  neural  network  concepts,  the  internal 

region  of  the  box  can  be  considered  as  the  'neural  world.”  For  a  neural* 
network  inference  machine  to  be  practical,  it  should  have  the  following 
functional  characteristics: 

1.  The  operational  goal  of  the  system  is  to  infer  approximate 

conclusions  to  its  inputs  and  generate  outputs  quickly  and 

efficiently.  'Approximate*  conclusions  are  appropriate  here  because 

neural  networks  are  known  to  have  good  generalization  properties  but 

are  weaker  at  detailed  analysis.  The  tradeoff  is  the  gain  of  fault 

tolerance  and  speed  at  the  expense  of  accuracy.  If  a  more  exact 

analysis  of  the  situation  is  desired,  a  digital  computer  could  be 

employed  to  further  any  analyses. 

2.  The  inputs  can  convey  complete  or  incomplete  analog  information  in  that 

there  may  not  be  a  unique  mathematical  representation  for  a  particular 

input  that  is  feasible  to  generate  or  even  consider.  An  example 

problem  for  which  this  type  of  information  is  typical  is  pattern 

recognition.  A  mathematical  description  of  the  desired  pattern  cannot 

be  known  conveniently,  but  a  child  can  easily  recognize  the  pattern 

without  this  information. 

3.  The  system  should  be  an  asynchronous  continuously'driven  system  as 

opposed  to  being  solely  an  input-driven  or  'query-driven*  system  in  the 
terminology  of  Ref.  1.  That  is,  its  inference  processing  should  be 

able  to  proceed  in  the  absence  of  real-world  inputs.  Similarly,  all 

inputs  should  be  treated  as  asynchronous  functions  of  time. 

4.  The  system  should  be  capable  of  anticipating  or  expecting  future  inputs 
given  the  past  history  of  the  inputs  and  the  information  it  has 
accumulated. 

Finally,  since  several  inference  processes  involve  causality,  it  would  be 

useful  to  endow  these  systems  with  this  time-domain  feature.  Current  neural 
network  models  do  not  use  the  temporal  dimension  effectively  for 
processing.  ‘ 
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Fig.  2  Block  diagran  of  the  loferact.  Triaogics  represent  input/output 
neural  layers,  trapezoids  represent  input/output  processing  layers, 

and  rectangles  represent  Inference  processing  layers.  The  number  of 
arrows  between  two  layers  gives  a  relative  indication  of  the  number 

of  connections. 
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THE  INFERNET  SYSTEM 


With  the  above  considerations  in  mind,  we  have  conceived  the  Infernet  (for 

INFERence  NETwork),  an  architecture  designed  using  neural  layers  as  the 

fundamental  building  block.  A  block  diagram  of  the  Infernet  is  shown  in 
Fig.  2.  The  architecture  can  be  grouped  into  three  sections:  the  input 
processors,  the  hierarchical  inference  processor  (HIP),  and  the  output 

processors.  The  main  flow  of  information  processing  occurs  from  left  to 

right  in  this  figure.  Each  processor  with  each  section  is  composed  of 

several  neural  network  layers  (represented  by  trapezoids  in  Fig.  2,  with  the 
lengths  of  the  left  and  right  sides  of  a  trapezoid  indicating  its  relative 

number  of  inputs  and  outputs,  respectively).  To  understand  the  processing 

concept  of  the  Infernet,  it  is  more  convenient  to  begin  with  the 

input/output  structures  and  then  proceed  to  the  HIP. 

The  Infernet  interacts  with  the  real  world  via  external  inputs  and 
external  outputs.  The  input  processors  are  responsible  for  sensing  and 
analyzing  the  external  input,  Sj,  i-l,2,...,N  to  produce  a  vector  of 
neural  signals  X|  for  the  HIP.  For  external  inputs,  there  must  be 

corresponding  input  processors. 

In  general,  each  input  processor  consists  of  three  layers,  as  shown  in  Fig. 

3.  The  first  layer  (shown  as  a  left>pointing  triangle  on  the  left  side  in 
Figs.  2  and  3)  senses  the  external  stimuli  relevant  for  the  particular 

external  input  and  produces  a  representative  set  of  neural  signals.  In 

effect,  this  first  layer  performs  an  input  format  conversion  between  the 
real  world  and  the  neural  world  of  the  Infernet.  The  second  layer 

decomposes  the  neural  signals  into  a  basis  set  of  information  building 

blocks  on  which  ail  analyses  can  be  performed.  ‘  As  this  step  usually  can 

involve  a  reduction  in  the  volume  of  input  data,  the  output  side  of  the 
second  layer  is  shown  in  Fig.  3  as  being  smaller  than  the  input  side.  In  an 

actual  implementation,  the  first  two  layers  may  be  combined  because  data 
reduction  may  not  be  necessary  for  all  input  stimuli  of  interest. 

The  third  layer  processes  the  basis  set  by  analyzing  various  collective 
features  to  expand  the  information  available.  Since  several  different 
analyses  may  be  performed  by  this  third  layer,  the  output  side  is  shown 
larger  than  the  input  side.  The  output  of  the  third  layer,  xj,  is  the 

output  of  the  i^”  input  processor,  which  corresponds  to  the  extetnal  input 

Sj.  The  combined  outputs  x^,  i-l,2,...,N  jq  comprise  all  of  the  inputs  to 

tne  HIP  that  originate  in  the  real  world. 

For  all  three  layers,  the  actual  processing  operations  are  determined  by  the 

topology  of  the  interconnections  and  their  associated  weights.  When  the 
weights  are  allowed  to  vary,  the  input  processors  can  adapt  to  new 
reductions/analyses  with  time. 
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Fig.  3  An  input  processor  for  the  infernet 


The  output  processors  perform  the  compiementary  operation  of  the  input 
processors.  They  convert  a  portion  of  the  neurai  output  signals  yj,i- 
from  the  HIP  into  action  responses  rj  for  the  real  world.  For 
Nou|  external  outputs,  there  must  be  the  same  number  of  output  processors. 
As  shown  in  Fig.  4,  each  output  processor  consists  of  two  layers. 
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Fig.  4.  An  output  processor  in  the  Infernet 
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The  first  layer  transforms  the  HIP  signals  yj  into  the  basis  set  of 
coordinated  response  signals  for  the  real  world.  The  second  layer  actually 

generates  the  real-world  actions  r^  corresponding  to  these  response  signals. 

As  with  an  input  processor,  the  actual  reduction  operations  and  output 

transformations  are  determined  by  the  interconnection  topology  and  strengths 
within  the  two  layers.  When  the  strengths  are  allowed  to  vary,  an  output 
processor  can  adapt  to  produce  new  or  improved  external  output  actions  with 
time. 

As  indicated  in  Fig.  2,  the  external  outputs  possibly  can  influence  the 

external  inputs  via  feedback  in  the  real  world  (represented  by  the  feedback 

function  F(r)  where  r  is  the  vector  of  external  outputs  r^).  It  is  believed 

that  this  feedback  path  is  very  important  in  the  learning  process, 

particularly  for  inductive  learning.  One  of  the  tasks  of  this  program  is  to 
investigate  the  effects  of  real-world  feedback  on  the  ability  of  the 

Infernet  to  assimilate  new  knowledge. 

Functionally,  both  the  input  and  output  processors  are  input-driven 

structures  in  that  they  produce  no  output  signals  in  the  absence  of  the 

appropriate  inputs  (or  stimuli).  In  this  sense,  they  represent  static 

processing  sections. 

The  HIP,  on  the  other  hand,  is  a  dynamic  processing  section.  Connecting  the 
output  processors  with  the  input  processors,  the  HIP  is  the  inference  engine 
of  the  Infernet  and  thus  contains  the  knowledge  base.  Because  of  its 

dynamical  nature,  the  HIP  operates  continuously  to  allow  for  further 
inferring  and/or  processing  on  the  knowledge  it  has  stored.  This  ability 

permits  the  Infernet  to  continue  processing  inputs  even  after  the  driving 

stimuli  have  been  removed  and  the  input  processors  have  become  silent. 

An  expanded  block  diagram  of  the  HIP  is  shown  in  Fig.  5.  Since  the  HIP  is 

composed  of  neural  layers,  the  information  contained  in  the  knowledge  base 
is  stored  in  a  distributed  manner  according  to  the  topology  of  the 

interconnections  between  the  neurons  and  their  associated  synaptic 
strengths.  Consequently,  Fig.  5  does  not  show  any  explicit  memory 

structures  (i.e.  layers  whose  sole  function  is  information  storage)  within 

the  HIP. 

The  HIP  consists  of  several  levels  of  neural  layers  arranged  in  a  linear 

fashion  such  that  any  particular  level  only  can  affect  its  two  neighboring 
levels.  The  configuration  shown  in  Fig.  S  depicts  three  levels  in  a 
vertical  format.  The  lower  level  of  this  chain,  labeled  "Level  1",  accepts 

the  processed  input  signals  x:,  ial,2,...,N  jg,  from  the  input  processors  and 

also  receives  the  signal  Hj,  trom  the  next  higher  level,  labeled  "Level  2". 
The  first  level  processes  these  inputs  by  mixing  them  using  several  neural 

layers.  The  resulting  outputs  form  the  second  level  signal,  u ,  j  <  and  the 

response  signals  yj,  i>l,2,...,N  which  are  sent  to  the  output 

processors. 


9 


Internal 
Input  tt,n 


Ltvtl  3 
Monitor 
Signal 


Level  1 
Gtddanee 
Signal 


Level  3 

Internal 
Output  Uovt 


Level  2 
Gnidanee 
Signal 


Uji 


From 

Input 

Processors 


Level  1 


Ull 


Level  1 
Monitor 
Signal 


yi 


YffOv, 


To 

Output 

Processors 


Fig.  5  Ab  cxpaadcd  block  diagram  of  tbe  hierarchical  inference  processor 
(HIP)  of  the  Infernet.  In  this  figure,  each  connection  arrow 
represents  a  vector  of  neural  signals  and  each  rectangle  corresponds 
to  a  neural  layer.  Thus,  Level  1  is  composed  of  several  layers 

while  Levels  2  and  3  consist  of  single  layers.  The  triangles  for 

the  internal  input  and  output  signals  indicate  that  they  can  undergo 
a  transformation  to/from  the  neural  world  of  the  HIP,  depending  upon 
their  desired  form.  Note  that  the  structure  has  no  explicit  memory 
blocks  or  components;  knowledge  is  stored  distributively  throughout 
the  HIP  levels. 


As  envisioned,  the  first  level  in  this  chain  serves  as  the  main  decision* 

maker  of  the  Infernet,  and  for  this  role  it  receives  a  guidance  signal  u^^ 

from  the  second  level.  In  this  context,  Uj,  can  be  viewed  as  an 

expectation  signal  which  can  be  used  to  set  the  state  of  the  first  level  to 

anticipate  certain  external  inputs.  The  feedback  signal  u^j  informs  the 

second  level  as  to  the  decisions  made  by  the  first  level.  In  this  way,  the 

second  level  acts  like  an  advisor  for  the  first  level  and,  in  an 

hierarchical  sense,  can  be  considered  above  the  first  level.  In  terms  of 
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expectation,  h  ,  ^  can  be  used  to  determine  what  the  second  level  should 

anticipate  from  the  third  level.  Another  benefit  of  the  *  *21  feedback 

path  is  that  it  provides  a  mechanism  for  the  HIP  to  perform  further 

processing  on  the  stored  information. 

The  idea  of  hierarchical  advising  now  can  be  applied  to  the  second  level. 
In  addition  to  the  first  level,  this  level  interacts  with  a  third  level  via 

the  monitoring  signal  and  the  guidance  signal  However,  signals 

between  Levels  1  and  3  are  not  permitted  because  they  would  eliminate  the 

hierarchical  structure  of  the  HIP.  This  arrangement  would  allow  a  "fully 

interconnected"  topology  between  the  levels  and,  thus,  would  undermine  the 
interaction  between  the  various  levels.  Since  this  interaction  is  believed 
to  be  important  in  creating  a  neural  network  inference  engine,  such 

interlayer  connections,  between  nonadjacent  levels  in  the  HIP,  are  not 

considered  here.  For  a  general  HIP,  the  advising  process  can  be  extended  to 
include  many  more  levels.  However,  the  Infernet  system  shown  here  consists 

of  only  three  levels  in  the  HIP  to  study  the  general  inference-making 

properties  of  such  an  architecture.  ^ 

Without  a  monitoring  level  with  which  to  interact,  the  third  level  has  .the 

potential  to  be  affected  by  some  other  source.  As  shown  in  Figs.  2  and  5, 

this  guidance  signal  is  called  the  internal  input  ujg;  and  correspondingly, 

the  third-level  monitor  signal  is  the  internal  output  The  "internal” 

aspect  of  the  guidance  signal  refers  to  the  fact  that  its  influence 

originates  at  the  upper  level  of  the  HIP  and  filters  down  to  the  lowest 

level.  One  possible  use  for  uj.  is  to  provide  a  form  of  external 

supervision  when  training  the  Internet.  In  the  same  way,  the  internal 

monitor  signal  provides  an  indication  of  the  activity  of  the  highest 

level  in  the  HIP.  Depending  upon  the  their  usage,  the  internal  input  and 
output  signals  may  be  desired  in  a  real-world  format,  which  is  illustrated 
in  Figs.  2  and  5  by  the  small  left-  and  right-pointing  triangles, 
respectively. 

Although  the  first  level  remains  the  primary  decision-making  layer,  it  is 
believed  that  the  entire  hierarchical  structure  is  necessary  to  make  the  HIP 
an  inference  engine.  The  first  level  receives  guidance  inputs  directly  from 

the  second  level  and  indirectly  from  the  third  level.  Thus,  to  a  certain 

extent,  all  levels  are  involved  in  the  decision-making  process.  It  is  this 
interlevel  advising  in  conjunction  with  the  anticipatory  guidance  signals 
that  is  believed  to  give  the  HIP  architecture  its  inference  and  expectation 

capabilities,  properties  which  must  be  validated  during  the  remainder  of 
this  research  program. 

As  shown  in  Fig.  5,  the  second  and  third  levels  each  are  composed  of  a 
single  neural  layer  whose  sole  function  is  to  combine  the  monitor  signal 
from  the  lower  level  and  the  guidance  signal  from  the  upper  level  to  form  a 
guidance  signal  for  the  lower  level  and  a  monitor  signal  for  the  upper 
level,  i^he  exact  method  by  which  these  two  layers  perform  this  task  depends 


n  addition,  a  ’fully  intareonnaetad*  natwork  plaeat  larga  damanda  upon  tha 
impiamantation  taehnolocx- 
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upon  their  intralayer  connection  schemes. 


In  contrast,  the  first  level  is  composed  of  several  smaller  layers  (or 

sublayers)  for  combining  the  outputs  of  the  input  processors  with  the  second 
level  guidance  signal.  One  possible  arrangement  for  these  sublayers  is 
shown  in  Fig.  5.  The  concept  of  this  arrangement  is  to  compare  and 
correlate  the  external  inputs  with  the  expectations  of  the  upper  levels  of 
the  HIP  and  then  decide  on  an  action  based  on  this  comparison.  Lastly,  the 
upper  levels  can  be  updated  as  to  any  decisions  made,  while  the  output 

processors  generate  the  corresponding  real-world  responses. 

Based  on  this  concept,  the  topology  of  the  first  level  sublayers  consists  of 

three  stages.  The  first  stage  compares  the  external  inputs  with  the  upper 

level  HIP  signals  by  individually  mixing  the  input  signals  X|  with  the 
guidance  signal  u,.  using  N|q  sublayers  to  produce  influence  signals  vj 

(see  Fig.  S).  The  second  stage  consists  of  a  single  sublayer  which 

generates  a  decision  signal  P  based  on  the  influence  signals.  Thus,  the 
second  stage  has  most  of  the  decision-making  power  of  the  first  level. 
Finally,  the  decision  signal  P  is  used  to  create  a  new  monitor  signal  u.j 

using  the  old  guidance  signal  in  one  of  two  sublayers  comprising  tne 

third  stage.  The  other  sublayer  decodes  the  decision  signal  P  into  the 
corresponding  action  signals  yj  so  the  output  processor  can  produce  the 
appropriate  real-world  responses. 


DYNAMIC  KNOWLEDGE  BASE  ENCODING 

As  previously  stated,  the  knowledge  base  is  stored  in  the  HIP;  and  since  the 
HIP  is  a  neural  network,  the  actual  storage  format  is  a  spatial  distribution 
of  synaptic  weights.  As  with  the  input  and  output  processing  stages,  the 
interconnection  strengths  within  the  HIP  layers  can  vary  with  time,  thus 
allowing  the  knowledge  base  to  be  altered  and  expanded  with  time. 

However,  there  is  a  significant  difference  between  a  knowledge  base  that  a 

human  can  understand  and  a  spatial  distribution  of  numbers.  As  a  result,  a 
formidable  problem  exists:  how  to  determine  the  spatial  distribution  of 
interconnection  weights  given  a  human-understandable  knowledge  base.  To 
address  this  problem,  a  solution  which  relies  on  the  dynamic  nature  of  the 
HIP  is  proposed  and  shall  be  referred  to  as  dynamic  knowledge  base  encoding. 
This  idea  represents  an  attempt  to  use  the  temporal  dimension  of  a  neural 
network  for  processing. 

In  this  scheme,  a  knowledge  base  consisting  of  data  objects,  relationships, 
facts,  and/or  rules  is  recognizable  as  temporal  "waves”  of  neuron  activity 
cycling  through  the  levels  of  the  HIP.  Each  piece  of  knowledge  (data 
objects,  relationships,  etc.)  is  stored  distributively  over  localized 
regions  of  the  HIP.  Many  waves  can  exist  simultaneously.  Associations 

within  the  knowledge  base  are  made  "visible"  by  monitoring  the  trajectory  of 

a  wave  within  the  HIP. 

The  role  of  the  interconnection  coefficients  becomes  one  of  indirectly 
storing  the  knowledge  base.  Rather,  they  represent  the  degree  to  which  a 

wave  along  one  trajectory  can  be  redirected  into  another  path,  indicating 
that  different  information  is  being  addressed. 
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The  inference  process  becomes  the  sequence  of  transitions  from  one  wave 
trajectory  to  another,  and  so  on  until  a  conclusion  is  generated.  In  the 
case  of  deductive  inference,  a  wave  is  simply  redirected  into  other  regions 
of  the  HIP  whereby  it  either  can  stimulate  new  waves  or  pass  through  a 
region  which  is  the  correct  response  to  the  inputs.  On  the  other  hand, 
inductive  inference  becomes  one  of  creating  access  to  previously 
unaccessible  regions  of  the  HIP  and  associating  particular  pieces  of 
knowledge  with  those  regions.  The  interaction  of  various  waves  within  the 
HIP  is  illustrated  in  Fig.  6. 


HIP 


Fig.  6  Neural  wave  interaction  within  the  HIP. 


Note  that  if  time  could  be  stopped  or  frozen,  the  knowledge  base  would  be 
"concealed"  in  that  the  past  trajectory  of  a  wave  would  not  be  visible  nor 
could  its  future  path  be  determined  easily.  Thus,  current  and  future  pieces 
of  knowledge  are  "hidden"  from  view,  although  they  do  not  go  away  because 
they  are  maintained  by  the  interconnection  distribution. 

The  idea  of  dynamic  knowledge  base  encoding  and  neural  waves  raises  several 
important  issues,  such  as  the  interconnection  topology  and  initial  weight 
distribution,  the  wave  time  scale,  and  the  relationship,  if  any,  between 
neural  waves. 


NEURAL  CELLS 

Although  the  Infernet  is  designed  using  neural  layers  as  the  basic  building 
blocks,  the  neuron  or  cell  is  the  fundamental  processing  unit  in  a  neural 
network,  the  general  structure  of  which  is  shown  in  Fig.  7(a). 
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.  7  The  relationship  between  the  neuron  inputs  {j(t),  the  neuron  state 
pit),  and  the  neuron  output  i|(t)  in  the  pulse  neuron  model.  The 
structure  of  the  ceil  is  shown  in  (a),  the  set  of  Inputs  overlaid  on 
a  common  time  axis  in  (b),  the  state  in  (c),  and  the  resulting 
output  sequence  in  (d).  The  state  is  shown  for  the  case  when  p(t) 
is  reset  to  xero  when  an  output  impulse  is  generated.  Note  that  the 
Jumps  in  the  state  correspond  to  the  input  Impulses  and  are  scaled 
by  the  interconnection  coefficients  Pj(t). 


A  cell  has  n  inputs  {-(t),  n  interconnection  weights  }iu  a  state  variable 
p(t),  and  an  output  rKt)  which  is  derived  from  p(t).  To  capture  the  dynamic 
nature  of  the  HIP,  we  decided  to  first  investigate  pulse  neurons  rather  than 
continuous-value  cells.  In  a  pulse  neuron,  the  output  is  a  sequence  of 

identical  pulses.  The  information  being  transmitted  is  contained  in  the 

timing  relationships  between  the  pulses.  In  contrast,  continuous-value 
neurons  have  continuous  outputs  that  can  have  any  analog  value  within  a 
given  range  (usually  from  either  -1  or  0  to  1). 

Pulse  neurons  have  several  advantages  over  continuous-value  neurons. 

Analogous  to  frequency-modulation  communication,  pulse  neurons  are  more 
tolerant  of  noise  inherent  in  the  network.  Similarly,  small  variations  in 
the  actual  pulse  shape  (amplitude  and  duration)  do  not  affect  the 
transmission  of  information.  Neural  waves  would  be  observed  as  regions  of 
increased  firing  frequency  which  propagate  through  the  network. 

The  output  of  an  ideal  pulse  neuron  can  be  described  by  an  impulse  train  in 

the  form 


,*<()= 


(1) 


where  tj  is  the  time  at  which  the  i*^  impulse  occurs,  x  is  the  total  number 
of  impulses  generated,  and  a  is  the  magnitude  of  each  pulse.  The  sign  of 

the  impulse  train  depends  upon  the  type  of  connection  that  the  neuron  makes 
with  other  cells. 

It  is  assumed  that  a  cell  can  make  only  one  type  of  connection  with  other 
neurons  (either  excitatory  or  inhibitory  connections,  but  not  both).  As  a 

result,  the  -f  sign  is  used  in  Eq.  (1)  for  an  excitatory  neuron  (i.e.  one 

that  makes  only  excitatory  connections)  and  the  -  sign  is  for  inhibitory 

neurons.  For  simplicity,  the  terms  ’excitatory  neuron"  and  "inhibitory 

neuron"  will  be  used  to  refer  to  neurons  that  make  only  excitatory  or 

inhibitory  connections,  respectively.  Note,  however,  that  a  neuron  can  have 

both  excitatory  and  inhibitory  inputs;  it  is  only  the  output  that  is 

constrained  to  be  a  certain  type. 

The  inputs  to  a  cell  can  be  described  by 


{,♦«)= (2) 

•  si 

where  represents  the  input  and  the  ±  superscript  indicates  the 

signal  is  from  an  excitatory  (+)  or  inhibitory  (-)  neuron,  respectively. 

Alw,  in  Eq.  (2),  the  i*"  impulse  time  for  the  j*"  input  is  given  by 

tjj^‘“^and  the  total  number  of  impulses  by  As  with  continuous-value 

neurons,  the  state  of  a  pulse  neuron  is  simply  tne  temporal  summation  of  the 

inputs  weighted  by  the  interconnection  coefficients  p:  and  can  be  expressed 
by, 
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(3) 


J=1 

where  p(t)  represents  the  summed  history  up  to  time  t  of  all  of  the  impulses 
that  were  received  since  the  neuron  started  operating.  In  this  way,  p(t)  is 
a  memory  or  state  variable  for  the  neuron.  From  Eq.  (1),  the  output  of  the 
neuron  is  given  by 


,*(»)  =  w 

where  represents  the  impulse  times  for  the  output  and  the 

appropriate  sign  is  taken  for  the  corresponding  type  of  output  (excitatory 

or  inhibitory).  In  a  pulse  neuron,  an  output  impulse  is  generated  whenever 
the  value  of  the  summed  inputs  crosses  an  integer  multiple  of  a  threshold  y. 
Alternatively,  this  statement  is  equivalent  to  saying  that  the  inputs  are 
summed  until  a  threshold  is  reached,  at  which  time  an  tmtput  impulse  is 

generated  and  the  'summer*  is  reset  to  zero.  Thus,  the  output  impulse 
time  is  related  to  the  input  impulse  times  by 

-(‘1“")  =  *1  (5) 


if  the  state  is  not  reset  after  an  output  impulse  is  generated  and  by 


P 


(6) 


if  it  is  reset.  From  this  relationship,  the  activity  of  a  neuron  can  be 
determined  by  knowing  the  state  p(t).  In  addition,  the  cell  output  can  be 
calculated  using 


(7) 

instead  of  the  time-based  sum  from  Eq.  (4).  Here,  int(x)  is  the  function 
which  returns  the  greatest  integer  less  than  or  equal  to  x.  This  form  does 
not  depend  on  the  output  impulse  counter  k  nor  the  input  impulse  times 

explicitly,  and  it  is  more  useful  when  layers  of  neurons  are  considered  and 

in  the  design  of  the  optical  implementation.  The  relationship  between 

{j(t),  p(t),  and  )7(t)  is  illustrated  in  Fig.  7(b)*(d). 

NEURAL  INTERCONNECTIONS 

Since  the  connection  type  (excitatory  or  inhibitory)  is  determined  by  the 
neuron  output  and  not  by  the  sign  of  the  interconnection  coefficients 

^j(t),  only  positive  values  for  uj(t)  need  be  considered,  as  evident  by  Eqs. 
(2)  and  (3).  For  simplicity,  the  range  of  possible  values  are  limited  to 
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a  range  which  is  convenient  to  use  in  optical  implementations, 

particularly  when  the  interconnections  are  made  with  a  holographic 
transmission  mask. 

For  adaptability,  the  interconnection  coefficients  are  allowed  to  vary  with 
time.  The  key  factors  for  determining  an  adaptability  relationship  for 
Infernet  neurons  are  (1)  being  able  to  form  long-term  memory  (LTM,  i.e. 
storing  a  knowledge  base  and  updating  it)  and  (2)  providing  a  method  for 
global  reinforcement  (i.e.  a  way  for  a  supervisor  in  the  real  world  to  tell 
the  Infernet  that  an  output  response  is  either  right  or  wrong).  With  LTM, 
the  coefficients  are  adjusted  in  proportion  to  the  correlation  of  changes  in 
the  activity  of  the  connected  neurons.  This  type  of  learning  is  a  form  of 

differential  Hebbian  learning  and  is  used  because  it  is  believed  to  be  more 
beneficial  in  a  network  in  which  predictive  or  inferential  abilities  are 
desired  [3,4].  In  the  case  of  global  reinforcement,  all  coefficients  are 

increased  (corresponding  to  positive  reinforcement)  or  decreased  (negative 
reinforcement)  according  to  a  reinforcement  signal  which  indicates  if  the 
right  conclusion  was  inferred  to  the  given  inputs.  This  signal  is  generated 
externally  by  a  real-world  supervisor  and  only  indirectly  affects  the 

operation  of  the  Infernet. 

Since  the  coefficients  are  continuous-valued,  the  adaptability  relationship 
meeting  the  criteria  can  be  described  by  a  differential  equation.  However, 
it  is  convenient  to  define  certain  functions  first  in  order  to  easily  define 

the  Hebbian  learning  characteristics.  Let 


=  the  number  of  impulses  in  fi{t)  over  the  past  Ti  time  units 

(8) 

=  /'  i'KO'if' 

h-Ti  a 

(9) 

^(b) 

=  the  change  in  S(f})  over  the  past  Ts  time  units 

(10) 

=  E(,)|,  - 

(ID 

These  functions  are  illustrated  in  Fig.  8. 

With  these  functions,  the  differential  Hebbian  learning  relationship  is 

given  by 

=  <IH.bb(,(<),(,(<).wW|  (12) 

^  ^  (13) 

where  ,j(t)  is  the  i***  input  to  the  neuron,  nj(t)  is  its  associated  synaptic 
coefficient,  and  «(t)  is  the  reinforcement  function  whose  sign  indicates 
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positive  [<0(t)  >  0]  or  negative  [o(t)  <  0]  reinforcement.  Since  a  real- 

world  supervisor  determines  oi(t),  its  form  can  be  arbitrary.  However,  one 
possible  function  is 


f-l-1  if  the  Infemet  is  correct 

0  if  no  correction  is  desired  (14) 

-1  if  the  Infemet  is  wrong 


nit) 


Ti 


Ti 


S(f))  at  time  t  =5 
£(f;)  at  time  t-Tt  s  2 
A(t})  at  time  t  =  S 


Fig.  8  Sample  plot  of  t)(t)  Illustrating  the  I(q)  and  A(q)  functions  used  in 
the  differential  Hebbian  learning  reltionsbip. 


Also  in  Eq.  (13),  the  time  constants  and  are  the  LTM  and  the 
reinforcement  time  constants,  respectively,  with  assumed  to  be  much  less 
than  Tf.  Finally,  W(pj)  is  a  window  function  which  restricts  the  range  of 
p:  to  DC  0<p|41.  Possible  candidates  for  W(p|)  include  a  normal  rectangular 
window  function  in  this  range,  sin°(sp|)  where  n  is  a  positive  integer,  and 
apj(l-p:)  where  a  is  a  positive  constant.  We  currently  are  using  the  latter 
form. 

The  interconnections  described  so  far  represent  the  normal  method  of 
interconnecting  neurons.  That  is,  a  neuron  receives  many  inputs  from  the 
outputs  of  other  neurons  and  sends  its  output  to  still  other  neurons.  To 
support  dynamic  neural  waves  in  the  HIP,  a  global  signaling  method  will  be 
necessary  to  provide  constant  stimulation  for  the  pulse  neurons.  Without 
global  signaling  of  some  kind,  the  HIP  would  become  a  static  structure  that 
would  cease  being  active  once  the  external  inputs  were  removed. 

To  implement  global  signaling  in  the  HIP,  simple  cases  are  being  considered 
whereby  all  cells  in  the  HIP  share  a  common  input  (with  perhaps  different 
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weighting  coefficients).  Rather  than  being  a  pulsed  input,  it  is  either  a 

constant  value  or  one  that  is  allowed  to  vary  slowly  over  time. 

Functionally,  the  global  input  acts  like  a  "neural  clock”  by  stimulating  the 
neurons  to  fire  on  a  regular  basis  in  the  absence  of  any  external  (real- 

world)  stimuli.  However,  the  firing  frequency  can  vary  between  neurons  and 
depends  upon  their  respective  global  weighting  coefficients.  A  global  input 

of  this  type  represents  a  distinct  departure  from  conventional  neural 

network  theory. 


NEURAL  LAYERS 

A  neural  layer  is  composed  of  many  neurons  and  can  have  several  inputs  and 
outputs.  The  general  form  of  a  layer  in  the  Infernet  is  shown  in  Fig.  9. 
The  layer  input,  x(t)  is  a  vector  containing  a  total  of  M:q  signals  from 
other  layers.  In  the  same  way,  there  are  signals  in  tne  b.yer  output 

vector  y(t).  The  layer  consists  of  N  neurons,  and  so  the  interconnections 
between  them  are  represented  by  an  N  x  N  intralayer  connection  matrix  )i|(t). 


x(0 - -  Ms(0 


Layer 

Input 


— ► ' 

— ► , 

y(0 


Layer 

Output 


Number  of  neurons: 

N 

Number  of  inputs: 

Min 

Number  of  outputs: 

M^ 

Input  veetor: 

Min  X  1 

Intralayer  connection  matrix: 

NxN 

Interlayer  connection  matrix 

NxMin 

State  vector: 

ffxl 

Neuron  output  vector: 

N  X  1 

Output  mapping  matrix: 

Pf  X 

Output  vector: 

Movt  X  1 

Learning: 

(Intralayer) 

(Interlayer) 

x(0 

«,(0 

=  /It  puk. 

*l(0  =  ±«5»nt[ip(t)j 
H 

y(t)  = 

=  dHebb(^(t),v(t),w(t)) 
=  dHebb{i|(t),x(t),w(t)] 


Fig.  9  General  form  for  a  layer  of  pulse  neurons  in  the  Infernet. 
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Likewise,  the  interconnections  between  the  layer  input  x(t)  and  the  N 
neurons  are  denoted  by  an  N  x  interlayer  connection  matrix  p^Ct).  The 

state  of  the  layer  is  given  by  the  vector  p(t)  and  is  composed  of  the  states 
of  the  N  neurons.  Correspondingly,  ^(0  is  the  vector  of  ^11  neuron 
outputs.  The  mapping  between  y(t)  and  n(t)  »»  given  by  y(t)  -  where 

H  is  an  N  X  binary  matrix  with  one  nonzero  element  per  column. 

The  equations  governing  the  dynamics  of  the  layer  are  the  vector  extension 
of  the  individual  neuron  equations.  Using  £q.  (3),  the  layer  state  equation 
is 


(Ko=r  <‘5) 

w  “00 

Alternatively,  the  state-reset  form  could  be  used  in  which  case  the  lower 
bound  on  the  integral  is  changed  from  (infinity)  to  the  time  at  which  the 
last  impulse  was  generated.  From  Eq.  (7),  the  neuron  outputs  are  given  by 


f|=^(t)  =  ±a^int(ip(t)) 


(16) 


where  the  sign  is  determined  by  the  type  of  output  and  the  int(x)  function 
is  applied  to  each  vector  element  xj.  The  intralayer  and  interlayer 
connection  coefficients  vary  according  to  the  differential  equation  given  in 
(13).  In  matrix  form. 


and’ 


d 


dH«bb[9(«),ir(t),w(0) 

A(9)  9  A(9)  ^  w(Oq 

n  r. 


=  dHebb(i|(t),x(t),w(0] 


(17) 

(18) 

(19) 

(20) 


where  A(i})  is  the  vector  whose  elements  are  A(i){)  (i.e.  the  function  applied 
to  the  elements  of  (n),  O  is  an  N  x  N  matrix  containing  all  I’s,  and  the 
symbol  •  denotes  outer  product.  * 

The  most  general  topology  of  a  layer  is  one  that  is  "fully  connected*  so 
each  neuron  can  communicate  with  every  other  cell.  In  this  case,  if  there 


A  =  bAc,  than  a.j  =  ^i*j- 
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are  N  neurons  in  a  layer,  there  are  a  total  of  N  interconnections  possible 

and  the  layer  can  have  at  most  N  inputs  and  N  outputs.  This  configuration 

permits  an  output  to  be  affected  directly  by  an  input  via  one  neuron. 
Consequently,  the  inputs  are  not  well  ‘mixed*  together  before  an  output  is 
generated.  Also,  the  propagation  of  dynamic  neural  waves  can  be  hampered  by 
such  high  connectivity. 

Therefore,  layers  in  the  Infernet  are  more  restricted  in  their  overall 

configuration  to  allow  for  more  processing  of  the  inputs  before  generating 

the  outputs.  Instead  of  making  N  connections,  each  neuron  makes  on  average 
connections  with  other  cells  in  its  neighboring  vicinity.  Here,  M|jj, 

Mg^|,  and  Mi  are  all  on  the  order  of  M  where  1  «  M  «  N.  This  arrangement 
creates  a  hi^  local  interconnection  density,  given  by  M^,  and  a  low  global 
interconnection  density,  given  by 


total  average  number  of  interconnectiona  in  a  layer 
total  number  of  possible  interconnections  in  a  layer 

NMl  ^  Ml 
~  N  ‘ 


(21) 

(22) 


With  this  type  of  topology,  the  intralayer  connection  matrix  has 

approximately  nonzero  elements  somewhat  clustered  together  (reflecting 

the  neighborhood  connection  idea).  Since  M£^  «  N,  tends  to  be  a  sparse 
matrix. 


As  defined  by  £q.  (22),  Mq  also  acts  as  a  figure  of  merit  for  the  amount  of 
processing  performed  by  me  layer.  It  also  is  important  in  characterizing 
aspects  of  dynamic  neural  wave  propagation  through  the  layer.  If  Mq  «  1, 
the  layer  has  many  neurons  between  the  cells  which  receive  the  layer  inputs 
and  the  ones  which  provide  the  layer  outputs.  Thus,  the  inputs  undergo  a 
relatively  large  amount  of  processing.  However,  if  Mq  *  1,  the  inputs  are 
processed  relatively  little  compared  to  the  former  case. 


THE  INFERNET  NEURAL  NETWORK 

Many  of  these  layers  are  interconnected  to  form  the  Infernet  architecture 
shown  in  Figs.  2-5.  Since  the  neural  signals  are  temporal  impulses,  the 
Infernet  performs  temporal  processing  on  a  spatial  distribution  of  signals. 
If  represents  the  state  of  the  network  (as  the  vector  of  all  neuron 

states),  the  overall  transfer  function  G(.)  of  the  Infernet  determines  the 
external  outputs  r  in  response  to  the  external  inputs  given  Poe|(0  aod  the 
knowledge  base  (the  spatial  distribution  of  interconnection  coefficients 
throughout  the  Infernet).  This  function  can  be  summarized  by 


r  =  G{p^,»). 


(23) 


In  conjunction  with  the  real-world  feedback  function  Fr  to  include  the 
effects  of  r  in  •,  Eqs.  (lS)-(23)  form  the  basic  mathematical  model  of  the 
Infernet. 
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SUMMARY  OF  INFERNET  FEATURES 


The  distinguishing  characteristics  of  the  Infernet  can  be  seen  best  by 
comparing  its  various  aspects  with  some  of  the  previous  work  on  artificial 

neural  networks  and  systems.  The  following  list  summarizes  the  significant 

conceptual  features  of  the  Infernet: 

1.  The  Infernet  is  a  system  that  can  exploit  causality.  That  is,  the 

information  being  processed  is  encoded  as  a  spatial  distribution  of 
temporal  signals.  The  Infernet  cannot  produce  any  outputs  if  time  is 

"stopped,”  although  all  neuron  states  then  can  be  observed. 
Furthermore,  it  is  not  necessarily  simple  to  deduce  what  the  Infernet 
will  do  next  given  a  measurement  of  the  neuron  states.  Thus,  the 
Infernet  is  active  only  when  its  neurons  are  producing  outputs  pulses. 

In  contrast,  most  other  neural  networks  are  designed  to  process  a 
spatial  distribution  of  signals  (i.e.  vectors)  such  that  time  can  be 
stopped  temporarily  and  the  vectors  still  provide  output  information. 

Some  of  them  only  have  valid  outputs  when  the  activity  of  the  network 

ceases. 

2.  The  Infernet  architecture  includes  real-world  inputs  s  and  outputs  r  as 

opposed  to  assuming  that  information  already  is  encoded  into  a  neural 
format  using  some  method.  In  addition,  the  effects  of  real-world 

feedback  are  available  to  the  Infernet  to  increase  learning  efficiency. 

3.  The  Infernet  is  a  specific  example  of  an  architecture  which  uses  layers 

as  the  fundamental  building  block  for  designing  a  system.  Furthermore, 

the  Infernet  architecture  is  not  just  a  serial  arrangement  of  layers 
but  a  two-dimensional  distribution  of  layers  with  many  feedforward  and 

feedback  paths.  Conventional  neural  networks  tend  to  be  a  linear 
sequence  of  layers  with  little  or  no  feedback  outside  the  layers. 

4.  The  inference  engine  of  the  Infernet,  the  HIP,  is  a  collection  of 

stacked  layers  that  communicate  via  monitoring  and  guidance  signals 
between  neighboring  layers.  This  interaction  between  these  layers  in 

the  Infernet  should  give  it  inferential  capabilities.  In  addition, 

access  ports  (the  internal  input  and  output)  are  available  to  a  real- 

world  supervisor  so  he/she  can  influence  and  observe  the  operation  of 
the  upper  levels  of  the  HIP  and  the  inference  process.  Other  neural 

networks  that  have  been  studied  have  not  been  based  on  this 

hierarchical  structure. 

5.  The  knowledge  base  is  stored  distributively  throughout  the  HIP  in  the 

interconnection  strengths  and  topology.  As  a  result,  the  HIP  does  not 
have  any  explicitly-defined  memory  structures  or  components  (like  the 

content-addressable  and  associative  memories  being  studied  by  other 
researchers). 

6.  Access  to  the  knowledge  base  is  via  dynamic  neural  waves  (temporal  flow 

of  increased  neuron  activity)  which  cycle  through  the  layers  of  the 

HIP.  Stopping  time  conceals  the  knowledge  base  in  that  it  is  not 

necessarily  simple  to  determine  what  the  Infernet  was  processing.  To 
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the  best  of  our  knowledge,  dynamic  knowledge  base  encoding  and  access 
is  unique  to  the  Infernet. 


7.  The  Infernet  can  anticipate  inputs  since  previous  input  stimuli  and/or 

current  dynamic  neural  waves  can  prime  the  state  of  the  network  in 
specific  regions  of  various  layers  so  these  regions  are  nearly  ready 
to  excite  new  waves. 

8.  The  HIP  operates  even  in  the  absence  of  any  external  inputs  because 

of  a  global  input  that  is  common  to  all  HIP  neurons.  Most  other  neural 

network  models  only  focus  on  the  select  electrical  signaling  method 
believed  to  be  used  in  the  brain. 

9.  The  learning  of  the  Infernet  can  be  unsupervised  and/or  supervised 

indirectly  via  the  global  reinforcement  signal.  Under  indirect 
supervision,  the  interconnection  coefficients  are  increased  when 
the  output  conclusions  are  correct  and  decreased  if  they  are  wrong. 
The  supervision  is  indirect  in  that  only  interconnection  strengths 
are  changed  and  not  the  physical  processing  actions  that  generated 
the  outputs.  In  addition,  when  a  knowledge  base  has  been  learned 

satisfactorily,  the  learning  procedure  can  be  halted.  Other  types  of 
neural  networks  use  unsupervised  learning. 

10.  An  Infernet  layer  is  not  "fully  interconnected”  intentionally  for 

three  reasons.  First,  it  allows  the  processing  characteristics  of 
this  type  of  layer  to  be  studied  which  hopefully  can  aid  in 
designing  a  network.  Secondly,  it  is  believed  to  be  necessary  for 
supporting  dynamic  neural  waves.  Finally,  this  type  of  interconnection 
arrangement  reduces  the  technological  demands  on  the  implementation. 
Other  research  approaches  permit  a  ’fully  interconnected”  network  as 
needed. 

11.  The  interconnection  strengths  in  an  Infernet  layer  can  have  only 

positive  values  between  0  and  1.  The  type  of  connection  (excitatory  or 

inhibitory)  is  determined  by  the  associated  output  neuron.  In  the 

other  models,  the  strengths  are  bipolar  real  numbers  (usually  without  a 
limit)  and  their  sign  determines  the  connection  type. 

12.  For  theoretical  analyses,  the  output  signals  of  Infernet  neurons  are 

idealized  to  be  continuous-time  6  functions  which  have  the  same 
amplitude.  Conventional  neurons  usually  have  either  continuous-valued 
outputs  (analogous  to  analog  communications)  or  binary-valued  output 

states  (which  can  convey  only  a  limited  amount  of  information). 

Rather  than  being  a  new  neural  model,  the  Infernet  represents  an 

architecture  which  is  intended  to  complement  the  work  of  other  neural 

network  researchers. 


SECOND- YEAR  GOALS 

Given  the  system  design  of  the  Infernet,  there  were  three  goals  for  the 
second  year  of  the  project: 
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1. 


Design  a  suitable  optical  implementation  for  the  Infernet. 


2  Perform  computer  simulations  on  the  Infernet  to  study  the  dynamic 
knowledge  encoding  scheme. 

3.  Develop  a  two>dimensional  photometric  system  for  use  with  our 

computerized  control  system  (CCS). 

Because  these  goals  are  relatively  independent,  they  can  be  worked  on  in 
parallel.  However,  an  actual  implementation  of  the  optical  design  from  the 
first  goal  would  have  to  wait  until  the  second  goal  confirmed  that  the 
Infernet  architecture  acts  as  an  inference  machine. 


SYSTEM  DEVELOPMENT 

Optical  Inplcneatationi  of  the  Infernet 

The  layer  is  the  basic  building  block  in  the  Infernet  because  it  is  the 

fundamental  structure  to  which  ail  neural  equations  apply.  Thus,  the  focus 

of  the  optical  implementation  is  to  realize  a  cascadable  layer.  This 

approach  is  completely  general  since  given  a  working  layer,  several  layers 

can  be  arranged  to  form  the  Infernet  network.  Alternatively,  the  entire 

network  can  be  considered  one  big  layer. 

Rather  than  simply  showing  the  final  design  of  our  opticaIly*implemented 

layer  (herein  referred  to  as  an  optical  layer),  it  is  insightful  to  start  at 

the  equation  level  and  build  up  to  the  optical  realization.  Mathematically, 

the  main  variable  to  consider  is  the  vector  state  variable  p(t).  Its 

scalar  elements  Pj(t),  i''l,2,...,N,  are  functions  of  two  independent 

variables,  i  and  t.  Since  optical  data  paths  are  two-dimensional,  the  state 

elements  can  be  represented  as  a  scalar  function  of  two  spatial  variables, 

t^/t,  as  shown  in  Fig.  10. 


Optic&l 

Information 

Region 


Fig.  10  The  spatial  coordinate  system  used  in  the  optical  data  path. 

The  region  within  the  dashed  box  contains  the  light 

distribution  representing  a  neural  variable. 
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Here,  corresponds  to  a  spatial  axis  representing  the  temporal  variable  t 
and  fi  corresponds  to  i  and  is  orthogonal  to  t^  The  layer  state  in  the 
space  domain  can  be  written  as  the  scalar  function  p(t^/S).  Note,  however, 
that  in  the  vector  state  variable,  i  is  a  discrete  index  whereas  fi  can  be  a 
continuous  spatial  variable.  This  flexibility  allows  an  optical  layer  to 
implement  continuum  of  neurons,  limited  only  by  the  resolution  of  the 
optical  components. 


The  same  coordinate  transformation  can  be  applied  to  i)(t)  and  x(t)  to  give 
Tj(t^,/S)  and  x(t^/S),  respectively.  In  the  same  manner,  the  interconnection 
matrices  K«(0  &Qd  1^(0  become  scalar  functions  of  the  three  spatial 

^Tnus,  liiijCO  rewritten  as 

'  *  nere  fi:  and  correspond  to  i  and  j. 


variables 
♦s 


M 1  (r,Pj,/»p)  and  n  whire 

respectively.^  For  reference,  these  transformations  are  summarized  below: 


Pit) 

i!i(0 

1(0 

Utit) 

(24) 


In  terms  of  these  new  spatial  coordinates,  the  state  equation  in  (15)  can  be 
rewritten  as 


/-OO  i-00 

where  the  second  integral  over  has  replaced  the  finite  sum  of  the  dot 
products  in  Eq.  (IS).  Note  also  that  the  /Sj  integral  extends  over  all  space 
(from  to  •»■*•).  In  practice,  this  range  is  reduced  to  the  finite  length  of 
one  side  of  the  two-dimensional  optical  data  paths. 

Equation  (25)  is  a  special  case  of  the  more  general  form  given  by 


/+00  ^+00 
>00  /->00 


(26) 


which  represents  a  space-variant  system  since  the  integrand  kernel 
H(t®i,Pi;t*Q,l*Q)  depends  upon  the  output  coordinates  (t*Q,/*Q)  as  well  as  the 
input  coordWtes  (t^i./Jj).  This  type  of  equation  can  be  implemented  using 
the  optical  setup  shown  in  Fig.  11.*  In  this  system,  the  amplitude 
distribution  of  the  light  incident  on  the  input  plane  is  given  by  n(t^j,^j) 
and  by  p(t’j|,/Sp  in  the  output  plane.  A  lens  with  focal  length  f  images  the 
two  planes  with  unity  magnification.  The  space-variance  is  introduced  by  a 
computer-generated  hologram  (CGH)  located  directly  in  front  of  the  lens. 
This  hologram  encodes  the  interconnection  matrix  jji(t®j,/Jj;t®Q,/JQ).  While 


^Th«  fubfcriptt  *i*  and  *o*  will  r«f«r  to  cortoin  input  nnd  output  plnnoi  of 
th«  optieni  Inyor,  ratpoctivoly. 

*Provid«  eartain  approximationa  ara  maintainad. 
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current  CGH  technology  docs  not  permit  the  interconnection  strengths  to  be 
varied  in  real  time,  this  optical  setup  computes  Eq.  (26),  making  it  useful 
for  fixed  (learned)  knowledge  bases. 


Incident 

Coherent 
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Output 
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Fig.  11.  A  general  tpace*variant  optical  imaging  system  which 
implements  Eq.  (26) 


For  computing  Eq.  (25),  the  design  in  Fig.  1 1  leads  to  the  optical 
architecture  shown  in  Fig.  12.  This  setup  is  composed  of  three  space- 
variant  imaging  systems.  In  the  first  one,  the  neuron  outputs  T)(t*j,pj)  arc 
present  at  input  plane  1  and  CGH  1  encodes  the  intralayer  connection  matrix 

imaging  path  has  x(t*j,Pi)  at  input  plane  2  and 
i’^i’^o^  stored  in  CGH  2.  Both  CGH  1  and  i  are  designed  so  these  paths 
sum  only  over  to  produce 


(27) 


and 


B  =  dpi 


(28) 


at  the  beam  splitter  (BS).  Here,  the  optical  data  paths  are  assumed  to  be 
within  -/it*j,  /ljS+/.  These  terms  then  arc  combined  and  processed  by  CGH  3 
which  simply  sums  over  t*j  up  to  t*„  to  give 


/<j 

^  A  +  Bdi*i 


(29) 


at  the  output  plane.  Note  that  -/  has  been  used  as  an  approximation  for  • 
(infinity)  in  Eq.  (29).  This  optical  system  thus  implements  an 
approximation  of  the  layer  state  equation  in  (25). 
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Fig.  12  The  fpace-vnrinnt  optical  Imaging  system  which  Implements 

the  layer  state  equation  In  Eq.  (2S). 


To  complete  the  optical  layer,  the  neuron  outputs  must  be  derived  from  the 
states  as  defined  by  Eq.  (16).  In  terms  of  the  spatial  coordinates 
(t^Q,/Sg),  the  neuron  outputs  are  given  by 

>!*«./».)  =  A)] .  (30) 

As  an  approximation,  the  integer  function  can  be  ignored,  resulting  in  the 
linear  relationship. 


(31) 

which  can  be  implemented  using  another  space-variant  imaging  system  as  shown 
in  Fig.  11  with  an  appropriately  design  CGH.  Alternatively,  rKt’p,/}^)  can 
be  computed  using  a  conventional  optical  processor  based  on  Fourier  optics 
[S].  In  this  setup,  the  Fourier  plane  filter  can  be  either  a  CGH,  a  half¬ 
plane  optical  density  wedge,  or  a  schlieren  knife-edge  filter.  These 
filters  are  oriented  perpendicular  to  the  t’p  spatial  axis  so  as  to 
calculate  the  derivative  in  the  t’^  direction  only.  While  all  of  these 
filters  approximate  a  true  differentiation  filter,  the  schlieren  technique 
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is  the  simplest  but  the  most  crude  in  that  it  more  closely  resembles  a  high 
pass  filter  rather  than  a  true  differentiator. 


A  more  exact  way  of  calculating  is  to  threshold  using  a 

two-dimensional  spatial  light  modulator  such  as  the  microchannel  spatial 

light  modulator  (MSLM)  [6,7]  and  then  compute  the  partial  derivative  along 

t^Q  with  a  conventional  optical  processor.  In  addition  to  being  more 
accurate,  this  method  allows  p(tQ,fiQ)  to  remain  in  the  interval  [0,y] 

thereby  eliminating  the  need  to  evaluate  the  integral  over  t’  in  Eq.  (16) 

starting  from  •«*. 
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Fig.  13  The  optical  implementation  of  an  lafernet  layer.  As  in  Fig.  12,  the 
distances  between  the  imaging  lenses  Lj,  Ljt  and  Lj  and  their 
respective  input  and  output  planes  are  2f.  In  the  conventional 
optical  processor,  the  distances  between  the  transform  lenses  and 
Lj  and  their  associated  input,  transform,  and  output  planes  are  f. 
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Using  the  MSLM  approach,  the  complete  optical  layer  is  shown  in  Fig.  13. 

The  layer  input  x(tj,/>j)  enters  the  system  at  the  layer  input  plane.  This 

signal  can  be  the  output  from  another  optical  layer  or  be  generated 

electronically  via  an  electrically-addressed  two-dimensional  spatial  light 
modulator  (such  as  the  electron-beam-addressed  MSLM  [8]).  Likewise, 

rKtQ,/!^)  exits  at  the  layer  output  plane  and  can  be  directed  into  the  input 
plane  of  another  optical  layer  or  measured  with  a  two-dimensional 
photodetector  array.  In  addition,  the  state  output  is  available  at 

the  state  output  plane.  Note  that  the  optical  layer  illustrated  here 

employs  two  MSLM’s,  one  to  threshold  the  layer  state  and  other  to  buffer  the 

layer  output,  and  uses  the  schlieren  technique  in  a  conventional  optical 

processor  to  approximate  the  partial  derivative  given  in  Eq.  (31). 

When  an  optical  layer  is  in  operation,  dynamic  neural  waves  would  appear  as 
regions  the  light  moving  around  the  readout  side  of  the  output  MSLM.  Waves 
may  be  exited  within  the  layer  or  enter  via  the  layer  input  plane.  They 

then  may  cycle  within  this  layer,  exciting  more  waves,  or  be  sent  to  other 
optical  layers.  Since  these  waves  are  of  a  temporal  nature,  they  do  .not 
appear  in  the  static  hardware  diagram  in  Fig.  13. 

An  alternative  to  the  general  optical  layer  shown  in  Fig.  13  is  the  simplied 

design  shown  in  Fig.  14.  The  key  simplifications  in  this  design  are  that 

the  ceils  are  continuous-valued  rather  than  being  pulse  neurons.  This 

eliminates  the  need  for  a  spatial  representation  of  the  time  axis  (t^)  and 

thus  allows  a  matrix  of  cells  to  be  processed  (instead  of  a  vector  of 

neurons  as  in  Fig.  13).  While  this  design  is  much  more  sensitive  to  noise, 

it  is  simpler  to  construct  and  operate  in  order  to  demonstrate  the  basic 
implementation  concepts  before  proceeding  to  the  more  complex  optical  layer 
in  Fig.  13. 

With  this  optical  implementation  for  a  layer,  the  role  of  the  electronic  CCS 
is  to  control  all  layers  via  their  MSLMs,  monitor  the  operation  of  the 

network,  and  interact  with  the  real  world.  In  this  latter  case,  the  CCS 

would  also  function  as  the  stimuli  sensing  and  the  data  reducing  layers  in 
an  input  processor  (as  shown  in  Fig.  3)  and  the  action  generating  layer  in 

an  output  processor  (shown  in  Fig.  4). 

The  goal  of  the  simplified  optical  layer  is  to  demonstrate  the  basic 

architectural  components  of  the  Infernet.  Schematically,  the  relationship 

of  the  simplified  optical  layer  to  the  Infernet  block  diagram  in  Fig.  S  is 
illustrated  in  Fig.  IS.  Thus,  for  demonstration  purposes,  it  can  act  as  the 

entire  first  level  in  the  HIP.  Then,  the  electrical-to-optical  input 

spatial  light  modulator  must  emulate  all  of  the  outputs  of  the  input 
processors  as  well  as  the  guidance  signal  from  the  second  HIP  level. 

Similarly,  the  optical  output  sensed  by  the  camera  represents  the  input 

signals  to  the  output  processors  and  the  monitor  signal  for  the  second  HIP 
level.  Because  the  focus  of  the  simplified  optical  layer  is  more 
specialized,  the  role  of  the  CCS  is  complicated  since  it  must  keep  track  of 
many  different  signals.  However,  the  added  complexity  to  the  CCS  is  small 
compared  to  the  benefit  of  being  able  to  demonstrate  more  quickly  an  optical 
implementation  of  the  key  aspects  of  the  Infernet. 
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Fig.  14  Simplified  version  of  the  optical  layer  shown  in  Fig.  13. 

Interconnection  CGH's  are  omitted  for  clarity.  Each  optical  path 
represents  the  firing  activity  of  a  matrix  of  cells. 


At  '  the  system  level,  there  are  primarily  two  limitations  common  to  these 
implementation  designs.  First,  the  use  of  CGHs  to  realize  the 
interconnections  does  not  permit  the  strengths  to  vary  with  time  and  so  they 
must  be  predetermined,  resulting  in  a  fixed,  nonadaptable  knowledge  base. 
However,  this  limitation  is  not  too  restrictive  in  that  an  experimental 
prototype  of  the  optical  architecture  still  can  be  built  and  studied  during 
the  third  year  of  the  program.  Secondly,  the  performance  of  a  layer  is 

limited  by  the  resolution  and  speed  of  the  spatial  light  modulators  and  the 

number  of  electronic'to'optical  (and  optical-to-electronic)  data  conversions 
in  the  system.  In  the  general  optical  layer  (Fig.  13),  a  third  limitation 
exists  in  that  the  layer  inputs  must  be  shifted  along  the  t^:  axis  for 
continuous  time  operation.  Another  spatial  light  modulator  whicn  has  the 

needed  shifting  property  would  have  to  be  inserted  at  the  layer  input  plane 
to  accomplish  this  task.  Alternatively,  if  x(t’|,Pj)  is  generated  via  an 
electrically-addressed  spatial  light  modulator,  the  shifting  can  be 
performed  electronically.  However,  the  current  optical  device  technology 
still  can  be  used  to  demonstrate  the  implementation  designs. 
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Fig.  15  Rclatioaship  of  the  simplified  optical  layer  (Fig.  14)  to 

the  block  diagram  of  the  Infernet  (Fig.  5). 


COMPUTER  SIMULATIONS 

The  goals  of  the  computer  simulations  are  threefold:  (1)  to  understand  the 
dynamics  of  the  Infernet,  (2)  to  explore  various  methods  by  which  a 

knowledge  base  can  be  encoded  into  the  Infernet,  and  (3)  to  determine  the 

corresponding  interconnection  strengths  and  compute  the  required  CGHs  for 
the  optical  implementation  demonstrations. 

During  the  second  year  of  the  program,  several  simulations  were  carried  out 

to  study  the  various  aspects  of  the  Infernet  model.  The  primary  focus  of 

these  simulations  is  to  gain  insight  into  the  dynamics  of  a  general  layer. 

Early  simulations  showed  that  significant  computation  power  is  needed  to 
process  a  layer  with  only  tens  of  cells.  Consequently,  smaller  layers  are 
being  studied  to  limit  the  simulation  time  to  a  reasonable  value,  given  our 

computing  resources  (AT&T  3B2  running  Unix).  A  summary  of  the  major  results 
is  given  below. 
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With  no  global  input,  pulse  neurons  act  as  time*clomain  filters,  reducing  the 

number  of  pulses  being  produced  by  successive  cells.  Very  few  pulses  are 
produced  at  the  outputs  of  a  layer  even  when  there  are  many  pulses  at  the 

layer  inputs.  This  confirms  the  need  for  additional  (global  and/or 

continuous)  excitation  in  some  way.  With  a  constant  global  input  to  all 
cells,  a  background  firing  rate  is  obtained  with  the  intracellular 

connections  simply  varying  the  firing  rate  from  this  bias  value.  Unless 
stated  to  the  contrary,  all  later  simulations  have  the  equivalent  of  a 
continuous  global  input  that  is  spatially  uniform  and  constant  in  magnitude 

to  all  cells. 

For  a  dynamic  neural  wave  to  exist,  a  localized  region  of  increased  firing 
activity  (a  wave  "packet”)  should  be  able  to  exist  and  propagate  without 
spreading  out  over  the  entire  layer.  With  a  nearest-neighbor  interconnect 

scheme  (in  which  a  cell  makes  connections  with  only  its  closest  neighbors), 
any  wave  packet  eventually  disperses  over  time  and  ceases  to  be  localized 
and  propagating.  With  a  localized-feedforward  interconnect  scheme  (in  which 
a  cell  connects  to  its  neighboring  cells  only  on  one  side),  a  wave  packet 
disperses  and  forms  linear  wave  ’fronts”  (a  linear  wave  packet)  that  extends 

across  the  entire  layer  and  propagates  in  one  direction. 

Mathematically,  if  a  wave  packet  has  a  two-dimensional  envelope 

(representing  the  firing  rate  of  a  spatial  distribution  of  pulse  neurons) 
denoted  by  G(x,y)  at  time  t«0,  the  propagating  envelope  at  time  t  can  be 
expressed  as  G(x-v^t,  y-Vyt)  where  v^  and  v  are  the  velocities  along  the  x 
and  y  directions  and  are  functions  of  time.^  In  the  absence  of  dispersion, 
the  envelope  retains  its  shape.  However,  it  can  be  shown  that  the  envelope 
at  time  t-»-l  is  related  to  the  envelope  at  time  t  through  a  convolution  with 
the  weighting  vectors  of  the  ceils.  ^  For  no  dispersion  to  exist,  this 

implies  that  the  weights  must  be  set  such  that  G(x,y)  convolved  with  the 
weight  vectors  should  reproduce  G(x,y).  This  implies  the  weight  vectors  are 

impulses,  indicating  that  each  cell  makes  connections  with  only  onto  itself. 
If  both  G(x,y)  and  the  weight  vect'^rs  were  Gaussian  in  shape,  the 
convolution  would  result  in  another  Gau^mn  with  a  larger  variance,  thus 
effectively  producing  dispersion.  Hence,  the  computer  simulations  and 
mathematical  analyses  show  that  any  interconnect  scheme  will  produce 
dispersion,  thus  prohibiting  the  propagation  of  localized  regions  of  firing 
activity. 

As  originally  envisioned,  a  neural  wave  was  a  representation  of  neural 

activity  within  the  layer.  Given  the  dispersion  problem,  an  alternative 
wave  definition  was  investigated  to  see  if  it  had  the  desired  properties. 
Rather  than  looking  for  the  firing  activity  to  follow  a  wave  envelope,  we 
decided  to  look  for  possible  wave-propagation  behavior  in  the  continuous 
global  input.  Previously,  this  input  was  constant  throughout  the  layer. 
Now,  it  was  changed  to  have  a  spatial  Gaussian-like  distribution  whose 
position  could  be  affected  by  the  neural  firing  activity  within  the  layer. 

The  net  effect  was  that  the  cell  activities  were  the  wave  shape  plus  a 
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perturbation  term  due  to  the  interconnection  structure.  The  position  of 
this  "global  input  wave”  was  monitored  by  following  the  central  peak. 

Primarily  three  different  schemes  for  the  interaction  between  the  cell 
activities  and  the  global  input  wave  were  examined:  (1)  net  wave  gradient, 
(2)  potential  field,  and  (3)  variable  deflection.  These  methods  are 

classified  by  the  way  in  which  the  cells  could  alter  the  wave’s  trajectory. 
In  the  first  case,  the  wave  acceleration  in  a  particular  direction  was 

proportional  to  the  sum  of  the  cell  activities  weighted  by  the  gradient  of 
the  wave  envelope  in  that  direction. 

In  the  second  method,  the  cell  activities  acted  as  the  drive  to  create  a 
potential  field  which  could  create  forces  (and  thus  accelerations)  on  the 
wave.  This  approach  is  analogous  to  treating  the  wave  as  an  electron  in  a 
quasi-static  electric  potential  distribution  and  analyzing  the  system  using 

Newtonian  mechanics.  Both  of  these  methods  produced  relatively  inflexible 
layers  in  that  it  was  difficult  to  control  the  resulting  wave  trajectories 
and,  more  importantly,  the  coupling  between  trajectories. 

In  the  third  simulation,  the  cell  activities  followed  the  global  input  y/ave 
envelope  exactly  and  the  speed  of  the  wave  was  held  constant.  The 
interconnect  structure  served  to  create  a  spatial  distribution  of  angular 
deflections  which  could  change  the  direction  of  the  wave  trajectory.  The 
actual  deflection  at  time  t  was  the  accumulated  deflection  at  the  center 
peak  of  the  wave  (variable  deflection  throughout  the  wave  was  not  permitted 
as  this  would  deform  the  its  shape).  Unlike  the  previous  two  trials,  this 
formlation  resulted  in  a  layer  that  offered  significant  interaction  between 
wave  trajectories  and  allowed  localized  limit  cycles. 

Unfortunately,  the  layer  was  also  a  highly  chaotic  system  -  any  small 
perturbations  in  the  initial  conditions  produced  very  different 
trajectories.  This  is  most  undesirable  for  a  system  which  is  supposed  to  be 

fault  tolerant!  While  it  was  expected  that  a  wave  in  the  Infernet  might 
take  slightly  different  paths  every  time  the  system  responsed  to  the  same 
input,  it  is  necessary  that  the  system  should  consistently  produce  the 
correct  response  (after  learning). 

Once  again,  we  rethought  our  definition  of  a  wave,  keeping  in  mind  our  past 
simulation  experiences.  From  the  beginning,  we  viewed  a  general  Infernet 
layer  as  a  medium  capable  of  supporting  and  propagating  localized  regions  of 
increased  neural  activity.  However,  the  simulations  and  associated 
mathematical  analyses  did  not  support  this  behavior.  An  alternative  to  this 

view  is  to  consider  the  layer  as  a  resonator.  Different  wave  trajectories 

become  different  resonator  modes  and  the  transition  from  one  trajectory  to 
another  is  the  same  as  a  transition  from  one  mode  to  another  mode. 
Similarly,  a  limit-cycle  trajectory  maps  to  a  single  resonator  mode. 

In  order  to  see  how  a  layer  can  act  as  a  resonator,  consider  HIP  Level  1  to 

be  a  single  layer,  HIP  Level  2  to  provide  unity  feedback,  and  HIP  Level  3  to 

be  nonexistent.  Then,  the  outputs  of  Level  1  are  effectively  fed  back  into 

the  inputs  of  Level  1  (ignoring  the  input  processors  for  now).  A  mode 

becomes  any  input  distribution  to  Level  1  which  produces  the  same 
distribution  at  the  output  of  Level  I.*  To  determine  how  the  layer  forming 
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the  first  level  depends  upon  the  neural  network  model,  there  are  several 
different  models  are  available  (for  example,  back-prop  [9,10]  and  adaptive 
resonance  theory  [11-14]).  Because  of  their  connectionistic  origins,  a  mode 
is  relatively  fault  tolerant  and  stable.  Therefore,  by  using  these  ideas,  a 
resonator  theory  of  inference  may  be  investigated  in  the  context  of  the 
Infernet.  We  are  currently  examining  this  approach. 


TWO-DIMENSIONAL  PHOTOCALIBRATION  SYSTEM 

While  working  on  the  various  computer  simulations  of  the  Infernet,  we  also 
developed  a  two-dimensional  photocalibration  system  [15]  for  our 
computerized  control  system  (CCS).  The  objectives  of  this  subsystem  were 
twofold: 

1.  Provide  fast,  analog  measurements  of  a  general  two-dimensional  grid  of 
light  beams  in  the  presence  of  background  light  (noise)  and  with 
random-access  selection  (as  opposed  to  raster  scanning). 

2.  Allow  the  grid  parameters  to  be  easily  changed  and  have  automatic 
recalibration. 

The  second  objective  gives  our  photocalibration  system  considerable 
flexibility.  It  permits  the  size  of  the  grid  to  change  as  well  as  the 
orientation  while  keeping  the  necessary  overhead  parameters  transparent  to 
the  measurement  process.  It  also  allows  any  inversions  of  the  axes  due  to 

lenses  and  mirrors  to  be  accounted  for  regardless  of  the  actual  optical 
setup. 

The  only  hardware  required  is  a  vidicon  camera  and  a  frame  grabber  board  for 

our  AT&T  PC  6300  (the  central  component  of  the  CCS).  Consequently,  the 

above  objectives  are  realized  in  software.  For  our  system,  we  used  a  CCD- 
array  camera  and  a  PCVISlONplus  Frame  Grabber  board  with  the  ITEX  PCplus 
library  of  image  processing  routines.  Our  software  was  written  in  Microsoft 
C  version  5.0.  There  are  two  sets  of  routines,  an  initialization  module  for 

the  -  second  objective  and  a  runtime  module  for  the  first  one.  Given  the  size 
of  the  desired  grid  (say,  N  x  M),  the  initialization  module  determines  the 

position  and  angle  of  rotation  of  the  grid  in  the  camera’s  field  of  view. 
Effectively,  the  module  produces  a  mapping  between  the  pixel  coordinates  of 
the  optical  system  output  and  the  pixel  coordinates  of  the  camera  output. 
This  task  is  accomplished  by  operating  the  optical  system  and  processing 
four  camera  pictures.  First,  the  optical  output  is  erased  and  the  first 
picture  of  the  background  light  distribution  is  taken.  Next,  the  optical 

system  is  operated  such  that  it  produces  a  beam  of  light  at  the  (1,1)  grid 

location  in  the  output  and  a  picture  (#2)  of  the  resulting  image  is  taken. 
By  comparing  pictures  #1  and  #2,  the  initialization  module  can  determine  the 
location  of  (1,1)  for  the  grid. 

Then,  without  clearing  the  optical  output,  the  optical  setup  is  operated 

such  that  pixel  (1,M)  is  illuminated  while  leaving  the  light  at  (1,1) 
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iotftct.  This  ioQftgc  is  cspturcd  io  picture  #3  sod  then  coinpftred  to  picture 
#2  to  determine  the  (1,M)  grid  location  in  the  camera’s  field  of  view.  This 

also  allows  the  angular  rotation  of  the  grid  rows  to  be  calculated. 

Finally,  the  light  beam  at  (N,l)  is  produced  via  the  optical  system  and  a 
picture  (#4)  taken.  As  with  pictures  #2  and  #3,  pictures  #3  and  #4  are 
compared  to  determine  the  (N,l)  pixel  location  and  the  angular  rotation  of 

the  grid  columns.  An  interesting  aspect  of  this  algorithm  is  that  the 

optical  grid  pattern  does  not  have  to  be  rectangular;  the  grid  only  needs  to 

form  a  parallelogram.  Also,  there  can  any  number  of  row  or  column 
inversions. 

With  these  measurements,  the  initialization  module  computes  all  of  the 
necessary  values  needed  by  the  runtime  module  to  translate  a  grid  coordinate 
(n,m)  into  a  position  in  the  camera  field  of  view.  Then,  the  maximum  pixel 

radius  for  a  grid  elements  (assuming  a  circular  shape)  is  calculated  from 

the  size  of  the  grid  and  the  resulting  row  and  column  spacings  between  the 

grid  elements. 

The  routines  contained  within  the  initialization  module  are: 


Routine 

set__up__slm_mapping 
find___slm___zero _ ^zero 

f  i  n  d___slm__ma  xx_zer  0 

f  i  nd_sl  m^zer  o_maxy 

calculate  _j7arttals 

save__slm_^map__data 

load__slm__map__data 

on  off  and  size 


Dcscrlntlon 

Turns  on  and  initializes  the  frame  grabber,  and 
captures  the  reference  picture  (#1). 


Takes  picture 
element. 

#2 

and 

locates 

the 

(M) 

grid 

Takes  picture 
element. 

#3 

and 

locates 

the 

(1.M) 

grid 

Takes  picture 
element. 

#4 

and 

locates 

the 

(N,l) 

grid 

Converts  the  i 

information 

obtained 

from 

the 

above 

four  routines 
values. 

into 

the 

mapping  transformation 

Saves  the  mapping  transformation  parameters  to  a 
disk  file  so  that  the  optical  output  does  not  have 
to  be  remapped  every  time  the  CCS  is  turned  on. 

Reads  the  mapping  transformation  parameters  from  a 
disk  file. 

Returns  the  (x,y)  coordinates  in  the  frame  grabber 
fg  coordinate  system  of  a  particular  optical 
output  grid  element. 

Allows  a  pixel  representation  to  be  changed 
between  digital  (on  or  off)  and  analog  values. 
Also  allows  the  radius  of  pixel  to  be  set 
manually. 
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max_pixel radius  Determines  the  size  of  the  area  to  be  searched 

when  calculating  the  average  intensity  for  an 
optical  output  pixel. 

draw^mappiag  Produces  a  pixel  pattern  showing  the  locations  of 

alloptical  output  pixels  enclosed  by  two 
concentric  circles,  one  at  the  expected  radius  and 
the  other  at  the  maximum  radius.  This  image  can 
be  superimposed  onto  an  optical  output  image  to 
see  if  the  optical  system  has  been  moved  since  the 
mapping  was  last  calculated.  Useful  for 
diagnostic  purposes. 


Routines  for  capturing  pictures  with  the  camera  and  any  associated  image 
processing  routines  are  contained  within  the  ITEX  PCplus  library. 

Using  the  parameters  found  by  the  initialization  module,  the  runtime  module 
simply  determines  the  average  light  intensity  (encoded  from  0  to  127)  within 
the  maximum  radius  for  any  grid  element  upon  request.  To  use  the  runtime 
module,  the  optical  system  is  cleared  so  the  optical  output  is  erased  and  a 
background  picture  is  taken  for  reference  (to  reduce  noise  significantly). 
When  the  intensity  at  a  particular  pixel  is  desired,  another  picture  is 
taken  and  the  appropriate  routine  called  which  returns  the  value  of  the  grid 
element.  Other  locations  within  the  grid  may  be  accessed  as  required. 

There  is  only  one  major  routine  in  the  runtime  module  and  it  is  listed 
below: 

RfluHnt  Description 


get_avg__value  Returns  the  average  value  of  intensity  inside  the 

max__pixel_radius  area  for  a  given  grid  element. 

Together,  the  initialization  and  runtime  modules  satiny  the  desired 
objectives  outlines  above  and  realize  a  translation,  rotation,  and  scale- 
invariant  photodetection  system  for  measuring  two-dimensional  grid-like 
distributions  of  light.  This  system  completes  our  design  of  the  CCS  for  our 
optical  systems  based  on  spatial  light  modulators. 


FUTURE  WORK 

With  the  CCS  complete,  the  bulk  of  the  work  lies  in  theoretical  analyses  and 
computer  simulations  of  the  Infernet  architecture.  We  plan  to  continue 

researching  our  resonator  viewpoint  of  the  HIP  levels  and  of  encoding  a 

knowledge  base  into  such  a  system  using  this  mode  theory.  Once  the 

simulations  render  a  working  inference  machine,  a  set  of  CGH’s  will  be  made 

and  the  simplified  optical  layer  will  be  demonstrated.  Any  demonstrations 
of  the  general  Infernet  optical  layer  (based  on  pulse  neurons)  will  be 

foregone  until  the  simulations  demonstrate  the  advantages  of  this 

implementation  over  the  simplified  version. 
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